REMARKS 

Enclosed herewith is a Substitute Specification in which 
the specification as filed has been amended in various places to 
correct typographical and grammatical errors, and also to add 
section headings. 

In support of the above, enclosed herewith is a copy of 
the specification as filed marked up with the above changes. 

The undersigned attorney asserts that no new matter has 
been incorporated into the Substitute Specification. 

The claims have been amended to more clearly define the 
invention as disclosed in the written description. In particular, 
the claims have been amended for clarity. 

The Examiner has rejected claims 1-11 under 35 U.S.C. 
102(e) as being anticipated by U.S. Patent 6,310,982 to Allred et 
al. 

The Allred et al. patent discloses a method and apparatus 
for reducing motion artifacts and noise in video image processing 
in which an image signal is subjected to temporal filtering, where 
the amount of temporal filtering is adjusted in dependence on the 
amount of motion detected in the image signal. 

The subject invention relates to reducing noise in a video 
signal and combines a spatial filter with a temporal filter to 
perform this task. To that end, the spatio-temporal filter 
includes, as claimed in claim 1, "a sigma filter for reducing noise 
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in images" , wherein "the sigma filter comprises one filter kernel 
for operating on pixels from both a current image and from an 
output of the spatio-temporal filter unit, said output supplying a 
temporally recursive filtered image" . 

The Examiner equates the spatial filter of the claimed 
invention to the spatial filter 18 of Allred et al. and then 
indicates that a temporal filter 38 of Allred et al. operates on a 
kernel of pixels, and that the filtered image output is stored im 
memory 34 via filter functions 26 "which operates one pixel kernel 
array, where it is recursively processed again." 

Applicants submit that the Examiner is mischaracterizing 
Allred et al. in an effort to read the claims of the subject 
application thereon. In particular, the spatial filter 18 of Allred 
et al. does not "reduce noise in images". Rather, the spatial 
filter 18 filters a difference between the current image and a 
delayed version of the output of the temporal filter arrangement of 
Allred et al . This filtered output is not an image signal with 
reduced noise, but rather, a motion value M (col. 4, lines 42-43) . 
Further, there is no disclosure in Allred et al. of a kernel of the 
sigma filter receiving and operating on pixels of both the current 
image and from the output of the spatio-temporal filter. In fact, 
neither of the filters of Allred et al., as identified by the 
Examiner, operate on pixels of the current image or from the output 
of the filter. Rather, again, the spatial filter of Allred et al. 
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operates on the difference between the pixels of the current image 
and the output of the temporal filter Xo(t), while the weighted 
average or signal ratios module 38 (identified by the Examiner as 
the "temporal filter") receives the difference value and the motion 
value (from the spatial filter) and performs a mixing of these 
values based on control signals from the control. 

With regard to claim 2, the Examiner equate the spatial 
pixel buffer with the memory 34. However, claim 2 specifically 
states "a spatial pixel buffer (104) to storing pixels of the 
current image as supplied to the spatio-temporal filter unit" . 
Applicants point out that the memory 34 does not store pixels of 
the current image. 

The Examiner then equates the spatial pixel selector with 
the spatial filter 18 and filter functions 26. Applicants submit 
that the spatial filter 18 and the filter functions 26 clearly are 
not "for selecting pixels from the spatial pixel buffer" . 

The Examiner then equates the temporal pixel buffer with 
the memory 34 and the temporal pixel selector with the weighted 
average 38 and filter functions 26 and control 22. However, since 
none of these devices are connected to the output of the memory 34, 
then clearly these devices are not "for selecting pixels from the 
temporal pixel buffer" . 

Applicants would like to remind the Examiner that it has 
been well established that W A claim is anticipated only if each and 
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every element as set forth in the claim is found, either expressly 
or inherently described, in a single prior art reference." 
Verdegaal Bros. v. Union Oil Co. of California, 814 F.2d 628, 631, 
2 USPQ2d 1051, 1053 (Fed. Cir. 1987). 

As has been clearly shown above, Allred et al. neither 
discloses or suggests each and every element as set forth in the 
claims . 

In view of the above, Applicants believe that the subject 
invention, as claimed, is neither anticipated nor rendered obvious 
by Allred et al., and as such, is patentable thereover. 
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Applicants believe that this application, containing 
claims 1-11, is now in condition for allowance and such action is 
respectfully requested. 

Respectfully submitted, 




Edward W . Goodman , Reg . 28,613 
Attorney 

Tel.: 914-333-9611 

CERTIFICATE OF MAILING 

It is hereby certified that this correspondence is being 
deposited with the United States Postal Service as First-class mail 
in an envelope addressed to: 

COMMISSIONER OF PATENTS AND TRADEMARKS 
P.O. BOX 1450 
ALEXANDRIA, VA 22313-1450 



On V . t ^ t&o^ 

B y Q^A^j/^ 
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IN THE DRAWINGS 
Applicants propose to label the blocks in Figs. 1-3 of the 
drawings in accordance with the accompanying ANNOTATED SHEETS 
SHOWING CHANGES, These changes have been incorporated in the 
accompanying REPLACEMENT SHEETS. 



N:\UserPublic\GO\PX09GOC0.GOR 8 




SPATIO-TEMPORAL FILTER UNIT AND IMAGE DISPLAY APPARATUS 
COMPRISING SUCH A SPATIO-TEMPORAL FILTER UNIT 

BACKGROUND OF THE INVENTION 
Field Of The Invention 

[0001] The invention relates to a spatio-temporal filter unit, 
provided with a sigma filter, to reduce the noise in images. 
5 [0002] The invention further relates to an image display 
apparatus comprising such a spatio-temporal filter unit. 

Description Of The Related Art 

[0003] A unit of the kind described in the opening paragraph is 
10 known from G. de Haan et al, "Memory integrated noise reduction for 
television," in IEEE Transactions on Consumer Electronics, Vol. 42, 
No 2, May 1996, Pages 175-181. In this article^ a unit is described 
which is a cascaded system of two filters-? — , i.e., a spatial filter 
and a temporal ea ef ilter . The spatial and temporal filters together 
15 should result in effective noise reduction. In general, spatial 
filters require pixels from one image, while temporal filters 
require samples from two or more successive images. 
[0004] The spatial filter is a recursive sigma filter. It is 
based on a variant of the sigma nearest neighbors selection 
20 process. The combination process involves weighted averaging, which 
is optimal for additive white Gaussian noise. The weights of the 
described sigma filter can have three possible values, which are 
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selected based on thresholds that are adapted to the noise level. 
In the article^ also a noise estimator is described to estimate the 
noise level. 

[0005] The temporal filter is realized as a motion motion- 
5 compensated firot first- order temporal recursive filter. Therefore, 
this temporal filter requires a motion estimation and motion 
compensation system, which makes the concept expensive. 
Furthermore, a motion detection means is necessary to adapt the 
strength of the filtering when motion occurs. Otherwise^ the image 
10 is distorted. 

SUMMARY OF THE INVENTION 
[0006] It is a first object of the invention to provide a 
spatio-temporal filter unit, of the kind described in the opening 

15 paragraph^ that integrates spatial and implicit motion- compensated 
temporal noise reduction in one filter. For the motion 
compensation^ no motion vectors, or other means to express the 
amount and direction of movement, are required. 
[0007] It is a second object of the invention to provide an 

20 image display apparatus comprising such a spatio-temporal filter 
unit . 

[0008] The first object of the invention is achieved in that the 
spatio-temporal filter unit comprises one filter kernel designed to 
operate on pixels from both a current image and from the output of 
25 the spatio-temporal filter unit, being a temporally recursive 
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filtered image. The filtering is based on the sigma nearest 
neighbors selection. Pixels that have poor correlation with the 
original pixel are automatically discarded, whereas other pixels 
contribute to the output in proportion to their correlation. It 
5 involves weighted averaging of selected pixels from a predefined 3- 
dimensional window comprising horizontal, vertical and temporal 
directions. A major characteristic of this spatio-temporal filter 
unit is that the spatial and temporal filtering are done within the 
same filter kernel. This reduces the implementation cost 

10 significantly and leads to an easily optimized concept. Moreover, 
problems^ such as the so-called "dirty-window" effect or frozen 
noise, "comet tails," etc., are automatically solved since the 
temporal filter is no longer separable. Noise breakthrough at edges 
or around moving objects is also eliminated because the spatial and 

15 temporal filters complement each other. 

[0009] An embodiment of the spatio-temporal filter unit 
according to the invention comprises: 

- a spatial pixel buffer to otorc for storing pixels of the current 
image ; 

20 - a spatial pixel selector to oolQCt for selecting pixels from the 
spatial pixel buffer; 

- a temporal pixel buffer fee -for buffer ing pixels from the output 
of the spatio-temporal filter unit, said output being a temporally 
recursive filtered image; 
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- a temporal pixel selector fee -for select ing pixels from the 
temporal pixel buffer; and 

- the sigma filter comprising one filter kernel designed to operate 
on the pixels from both the spatial pixel selector and the temporal 

5 pixel selector, 

[0010] In the spatial pixel buffer, a 2 -dimensional spatial 
pixel selector is defined to select pixels to enable spatial 
filtering. In order to add temporally recursive filtered pixels in 
the filter kernel of the sigma filter, a second 2 -dimensional 

10 temporal pixel selector is defined in the temporal pixel buffer, 
which is filled with pixels taken from a picture memory. In this 
way, 3 -dimensional spatio-temporal filtering is realized within one 
filter aperture. The output of the sigma filter is stored in a 
picture memory and used for subsequent filtering. This is the 

15 temporal recursive part of the processing. Motion detection is 

inherent in the filtering. Each pixel is first compared with the 
original one to determine its contribution to the output. Where 
motion occurs, the pixel will have a large difference in grey gray 
value. Therefore, it will be automatically discarded. Furthermore, 

20 motion compensation is also inherent. This is realized by using a 
temporal pixel selector for the selection of the temporally 
recursive filtered pixels and a discriminating averaging filter, 
which uses a threshold. If there is no motion, the grey gray level 
correlation relative to the central input pixel is high around the 

25 central pixel of the previous image. When motion occurs, this 
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correlation is shifted in position automatically following the 
motion trajectory. Provided the motion is low enough, temporal 
filtering will take place because correlated pixels can be found 
somewhere within the temporal window, i.e.^ the temporal pixel 
5 selector. In other words, motion compensation is implicitly done, 
thereby making explicit motion compensation redundant. 
[0011] An embodiment of the spatio-temporal filter unit 
according to the invention comprises an adaptive sigma filter. The 
filter thresholds of the sigma filter are adjustable. That means 

10 the degree of filtering can be increased or decreased to suit the 
taste of the user of this embodiment of the spatio-temporal filter 
unit_j_ or of an image display apparatus comprising such a spatio- 
temporal filter unit. The thresholds for spatial and temporal 
filtering can be independently determined so that the desired 

15 balance between spatial and temporal filtering is achieved. 
[0012] An embodiment of the spatio-temporal filter unit 
according to the invention is characterized in that of at least one 
of the pixel selectors, being the temporal pixel selector and the 
spatial pixel selector, the aperture can be adjusted. This 

20 determines the strength of the filtering and the characteristics of 
the output. A combination of a narrow aperture and a large aperture 
with image alternation gives good results. 
[0013] An embodiment of the spatio-temporal filter unit 
according to the invention is characterized in that at least one of 

25 the pixel selectors, being the temporal pixel selector and the 
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spatial pixel selector, is designed such that the distance between 
the selected pixels is adjustable. The result is that the aperture 
of the selector can be adjusted while keeping the number of 
selected pixels more or less constant. The costs of the pixel 
5 selectors are based on their storage capacity, i.e.^ the number of 
pixels they can store simultaneously. The advantage of this 
embodiment according to the invention is that the costs for 
processing of the pixels can be relatively low. 
[0014] An embodiment of the spatio-temporal filter unit 

10 according to the invention comprises a motion detector designed to 
detect motion and to control the aperture of the temporal pixel 
selector based on detected motion. The aperture determines the 
strength of the filtering for still images and the range of the 
motion compensation for moving sequences. An advantage of such an 

15 embodiment is that the implicit motion compensation is optimized. 
The motion detector may detect global or local motion. As the 
aperture can be determined on a field basis, an optimal setting can 
be found for each sequence once the degree of motion activity is 
known. The detector does not have to provide information about the 

20 direction of movement. It is also possible that an external motion 
estimator provides information about the estimated motion. 
[0015] An embodiment of the spatio-temporal filter unit 
according to the invention comprises a motion estimator designed to 
supply motion vectors and to control the position of the temporal 

25 pixel selector related to the temporal pixel buffer, based on the 
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motion vectors supplied by the motion estimator. The advantage of 
such an embodiment is that the temporal proximity is increased. 
This is especially beneficial in the case of high object 
velocities. It is also possible that an external motion estimator 
5 provides the motion vectors. 

[0016] An embodiment of the spatio-temporal filter unit 
according to the invention comprises a noise estimator designed to 
estimate a noise level in the current image and to control the 
aperture of at least one of the pixel selectors, being the temporal 

10 pixel selector and the spatial pixel selector, based on the 

estimated noise level. As eaAdr -noted previously, the strength of 
filtering is related to the aperture of the pixel selectors. 
Relatively more filtering is allowed in case of a relatively high 
estimated noise level. Therefore^ optimal noise filtering is 

15 achieved at all noise levels. It is also possible that an external 
noise estimator provides the estimated noise level. 
[0017] An embodiment of the spatio-temporal filter unit 
according to the invention comprises a noise estimator designed to 
estimate a noise level in the current image and to control 

20 thresholds of the adaptive sigma filter based on the estimated 
noise level. The thresholds can be increased in case of a 
relatively high estimated noise level and decreased in case of a 
relatively low estimated noise level. Therefore^ optimal noise 
filtering is achieved at all noise levels. 

25 
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BRIEF DESCRIPTION OF THE DRAWINGS 
[0018] These and other aspects of the spatio-temporal filter 
unit and the image display apparatus according to the invention 
will become apparent from and will be elucidated with reference 
5 with respect to the implementations and embodiments described 
hereinafter and with reference to the accompanying drawings, 
wherein: 

[0019] Fig. 1 schematically shows an embodiment of a spatio- 
temporal filter unit; 
10 [0020] Fig. 2 schematically shows an embodiment of a spatio- 
temporal filter unit comprising a noise estimator, a motion 
estimator and a motion detector; and 

[0021] Fig. 3 shows elements of an image display apparatus. 

15 DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0022] Figure Fig. 1 schematically shows an embodiment of a 
spatio-temporal filter unit 100 and two memory units which are used 
by the spatio-temporal filter unit 100. These memory units might be 
shared with other units in a system comprising the spatio-temporal 

20 filter unit 100. The first memory unit is a picture memory 114 
for store storing the output of the spatio-temporal filter unit, 
being a temporally recursive filtered image. The second memory unit 
is a cache 116. The signal representing the images enters the 
spatio-temporal filter unit 100 at the connection 102. The filtered 

25 signal leaves the spatio-temporal filter unit 100 at the connection 
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118. Pixels from this filtered signal are also stored in the 
picture memory 114. Pixels from an image that enter the spatio- 
temporal filter unit 100 are buffered in the spatial pixel buffer 
104. The spatial pixel selector 106 selects pixels from the spatial 
5 pixel buffer 104 and provides these selected pixels to the filter 
kernel 107 of the sigma filter 112. Pixels from the picture memory 
114 are cached in the cache 116 for reasons of performance. From 
the cache 116^ pixels are buffered in the temporal pixel buffer 
108. The temporal pixel selector 110 selects pixels from the 

10 temporal pixel buffer 108 and also provides these selected pixels 
to the filter kernel 107 of the sigma filter 112. In this way^_ the 
filter kernel 107 operates on the pixels from both the spatial 
pixel selector 106 and the temporal pixel selector 110. 
[0023] Figure Fig. 2 schematically shows a relatively more 

15 sophisticated embodiment of a spatio-temporal filter 200. The white 
boxes in Figure Fig. 2 are part of the spatio-temporal filter 200. 
The shaded boxes in Figure Fig. 2 are not part of the spatio- 
temporal filter 200. These shaded boxes represent memory units. 
These might be shared with other units in a system comprising the 

20 spatio-temporal filter unit 200. The first memory unit is a picture 
memory 214 fee -for otorc storing the output of the spatio-temporal 
filter unit, being a temporally recursive filtered image. The 
second memory unit is a cache 216. The signal representing the 
images enters the spatio-temporal filter unit 200 at the connection 

25 202. The filtered signal leaves the spatio-temporal filter unit 200 
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at the connection 218. Pixels from this filtered signal are also 
stored in the picture memory 214. Pixels from an image that enters 
the spatio-temporal filter unit 200 are buffered in the spatial 
pixel buffer 204. The spatial pixel selector 206 selects pixels 
5 from the spatial pixel buffer 204 and provides these selected 
pixels to the filter kernel 207 of the sigma filter 212. Pixels 
from the picture memory 214 are cached in the cache 216 for reasons 
of performance. From the cache 216^ pixels are buffered in the 
temporal pixel buffer 208. The temporal pixel selector 210 selects 

10 pixels from the temporal pixel buffer 208 and also provides these 
selected pixels to the filter kernel 207 of the sigma filter 212. 
In this way^_ the filter kernel 207 operates on the pixels from both 
the spatial pixel selector 206 and the temporal pixel selector 210. 
The spatio-temporal filter 200 comprises a motion detector 224 

15 designed to detect motion and to control the aperture of the 
temporal pixel selector 208 210 based on detected motion. The 
spatio-temporal filter 200 further comprises a motion estimator 222 
designed to supply motion vectors and to control the position of 
the temporal pixel selector 210 related to the temporal pixel 

20 buffer 208, based on the motion vectors supplied by the motion 
estimator. The spatio-temporal filter 200 also comprises a noise 
estimator 220 fee -for estimate estimating a noise level in the 
current image and fee — for control ling the aperture of at least one 
of the pixel selectors, being the temporal pixel selector 210 and 

25 the spatial pixel selector 206, based on the estimated noise level. 
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Bcoidco that ln addition, the noise estimator 220 controls 
thresholds of the adaptive sigma filter 212, based on the estimated 
noise level . 

[0024] Figure Fig, 3 shows elements of an image display 
5 apparatus 300 according to the invention. The image display- 
apparatus 300 has a receiving means 302 for receiving a video 
signal representing the images to be displayed. The signal may be a 
broadcast signal received via an antenna or cable^ but may also be 
a signal from a storage device^ like a VCR (Video Cassette 

10 Recorder) or DVD (Digital Versatile Disk) . The image display 

apparatus 300 further has a spatio-temporal filter unit 3 04 for 
filtering the video signal and a display device 3 06 for displaying 
the images represented by the filtered video signal. The spatio- 
temporal filter unit 3 04 is implemented as described in Figure Fig. 

15 1 or Figure Fig. 2 . 

[0025] It should be noted that the above-mentioned embodiments 
illustrate rather than limit the invention and that those skilled 
in the art will be able to design alternative embodiments without 
departing from the scope of the appended claims. In the claims, any 

20 reference signs placed between parentheses shall not be constructed 
as limiting the claim. The word * comprising' does not exclude the 
presence of elements or steps other than those listed in a claim. 
The word u a" or u an" preceding an element does not exclude the 
presence of a plurality of such elements. The invention can be 

25 implemented by means of hardware comprising several distinct 
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elements and by means of a suitable programmed computer. In the 
unit claims enumerating several means, several of these means can 
be embodied by one and the same item of hardware. 
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ABSTRACT ^ OF THE DISCLOSURE 

Noise reduction is an important feature in consumer 
television. This is realized by spatial, temporal or spatio- 
5 temporal filters. Spatial filters require pixels from within one 
image, while temporal filters require samples from two or more 
successive images. The spatio-temporal filter unit (100) according 
to this invention integrates spatial and implicit motion- 
compensated temporal noise reduction in one filter. For the motion 

10 compensation^ no motion vectors are required. The spatio-temporal 
filter unit (100) is provided with a sigma filter (112) comprising 
having one filter kernel (107) designed to operate on the pixels 
from both a current image and from the output of the spatio- 
temporal filter unit, being a temporally recursive filtered image. 

15 The operation of the spatio-temporal filter unit (100) can be 

adjusted by varying the thresholds of the sigma filter (112) and 
the selection of pixels. The adjustments can be controlled by a 
motion estimator (222) , a motion detector (224) and a noise 
estimator (220) . 

20 

Fig. 1 
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